A Cognitive Approach to Sketch Understanding

Sketching is an interactive process of communication, using drawings and linguistic information to convey spatial and conceptual material. Our work on a computational model of sketching has the goal of both explaining human sketching and creating software that can be a human-like partner in sketching interactions. This focus has lead us to explore a very different set of tradeoffs from those typically chosen in multimodal interfaces. We highlight some results of our approach, including research performed using GeoRep, our diagrammatic reasoning architecture, and sKEA, a multimodal sketching tool used for knowledge acquisition in spatial domains.